NW-55

Unit test and CI

• added for all modules, subworkflows, workflows and functions.
• CI for linting and building Rust binaries and python ensembl-variation-utils package.

ensembl-variation-utils package

The nextflow bin were taking a lot responsibility of figuring out the file locations, talking with databases etc. These responsibility has been taken of and given to a separate python package called ensembl-varation-utils. It has 3 separate modules -

• clients: to establish connection and talking with clients. We currently have DB clients that talks to metadata and core databases but it can be replaced in future with gRPC or REST.
• file_locator: this module knows the internal /nfs and external ftp directory and file structure and can discover files such as FASTA, GFF, VEP cache etc. and do file operations.
• vep_config: similar to file_locator but specialised for VEP plugin and custom annotation data files.

VEP config generation

Instead of hardcoding everything to generate_vep_config.py script now we accept a JSON config which can be fashioned for each run with greater degree of variability.